feat(xai): add TTS connection pooling#1977
Conversation
🦋 Changeset detectedLatest commit: 01c2244 The changes in this PR will be included in the next version bump. This PR includes changesets to release 36 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| const XAI_WEBSOCKET_URL = 'wss://api.x.ai/v1/tts'; | ||
| const DEFAULT_VOICE = 'ara'; | ||
|
|
||
| export type GrokVoices = 'Ara' | 'Eve' | 'Leo' | 'Rex' | 'Sal'; |
There was a problem hiding this comment.
🟡 Default voice name doesn't match the declared voice type, misleading users into sending wrong casing to the API
The default voice is set to lowercase 'ara' (DEFAULT_VOICE at plugins/xai/src/tts.ts:25) but the GrokVoices type declares capitalized names ('Ara' | 'Eve' | 'Leo' | 'Rex' | 'Sal'), so users relying on type hints will send capitalized values that may be rejected by the API.
Impact: Users who pick a voice from the GrokVoices type (e.g. 'Ara') may get API errors or unexpected behavior if the xAI TTS endpoint requires lowercase names.
Inconsistency with the existing realtime module's voice type
The existing realtime module at plugins/xai/src/realtime/realtime_model.ts:23 defines GrokVoices = 'eve' | 'ara' | 'rex' | 'sal' | 'leo' (all lowercase), and its DEFAULT_VOICE = 'ara' matches. The new TTS module's GrokVoices type uses capitalized names while its DEFAULT_VOICE is lowercase 'ara', creating an internal contradiction. Since the voice value is passed directly as a URL query parameter at plugins/xai/src/tts.ts:438 (url.searchParams.set('voice', opts.voice)), the casing matters for the API call.
| export type GrokVoices = 'Ara' | 'Eve' | 'Leo' | 'Rex' | 'Sal'; | |
| export type GrokVoices = 'ara' | 'eve' | 'leo' | 'rex' | 'sal'; |
Was this helpful? React with 👍 or 👎 to provide feedback.
| void messages.close(); | ||
| } | ||
| }, | ||
| { signal: this.abortSignal }, |
There was a problem hiding this comment.
🔍 ChunkedStream does not pass timeout to withConnection
At plugins/xai/src/tts.ts:424, the ChunkedStream calls withConnection with only { signal: this.abortSignal } and no timeout. This means the connection acquisition uses the pool's default connectTimeout of 10 seconds (from agents/src/connection_pool.ts:73). In contrast, the SynthesizeStream at line 321 passes this.connOptions.timeoutMs. This inconsistency means the chunked stream ignores user-configured timeout for connection establishment, though the default 10s is reasonable.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
||
| export * as realtime from './realtime/index.js'; | ||
| export { STT, type STTOptions, type STTLanguages } from './stt.js'; | ||
| export { TTS, type GrokVoices, type TTSLanguages, type TTSOptions } from './tts.js'; |
There was a problem hiding this comment.
🔍 Two conflicting GrokVoices types exported from the same package
The package now exports two different GrokVoices types: one from tts.ts (capitalized: 'Ara' | 'Eve' | ...) at the top level, and one from realtime/realtime_model.ts (lowercase: 'eve' | 'ara' | ...) under the realtime namespace. While they don't technically conflict (different export paths), users importing both would encounter confusing type differences for the same concept. This should be unified.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
Verification
pnpm --filter @livekit/agents-plugin-xai buildpnpm --filter @livekit/agents-plugin-xai lint(passes with existing_utils.tswarnings)pnpm buildcue-clivalidation attempted with a temporary xAI TTS voice agent; dispatch reached the worker, but runtime TTS validation could not complete becauseXAI_API_KEYwas not available in the environmentPorted from livekit/agents#6317
Original PR description
Summary
ConnectionPoolto the xAI TTS plugin so streamed synthesis segments can reuse websocket connections.prewarm()through the pool and close pooled connections fromaclose().Fixes #6078
Root cause
The xAI TTS stream opened a websocket with
_connect_ws()for each synthesized segment and always closed it in_run_ws()cleanup. Unlike other websocket-backed TTS plugins, it did not use the sharedutils.ConnectionPool, so repeated streamed segments could not reuse an existing connection or be prewarmed.Notes
xAI documents the streaming TTS websocket as multi-utterance: after
audio.done, clients can send anothertext.delta/text.donesequence on the same connection.Test plan
uv run pytest tests/test_plugin_xai_tts.py --plugin xai -quv run pytest tests/test_realtime/test_xai_realtime_model.py --unit -quv run ruff check tests/test_plugin_xai_tts.py livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/tts.pyuv run ruff format --check tests/test_plugin_xai_tts.py livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/tts.pyuv run --group typing mypy --untyped-calls-exclude=smithy_aws_core -p livekit.plugins.xai